{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Lab 05: Finding Structure in Time
\n",
"\n",
"CS371: Cognitive Science
\n",
"Bryn Mawr College, Fall 2016
\n",
"Prof. Blank\n",
"
\n",
"\n",
"This is a team-based project."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the cell below, add your by-line, and delete this cell:"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbgrader": {
"grade": true,
"grade_id": "by-line",
"locked": false,
"points": 5,
"solution": true
}
},
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 1. Predicting the next word"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this lab we will attempt to replicate the results from \"Finding Structure in Time\":\n",
"\n",
"https://crl.ucsd.edu/~elman/Papers/fsit.pdf\n",
"\n",
"First, we need some text. For this demo, I'll make up a short text. For your assignment, you should generate sentences like Elman did in his paper. You should write a program that will generate random sentences of the appropriate grammar."
]
},
{
"cell_type": "code",
"execution_count": 218,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"text = (\"me like you you like me me like apples me like bananas \"\n",
" \"you like bananas you like apples you hate berries me \"\n",
" \"like berries me need berries you need apples you need me\").strip()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we write some encoding and decoding functions:"
]
},
{
"cell_type": "code",
"execution_count": 219,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"text_words = text.split(\" \")\n",
"\n",
"words = list(set(text_words))\n",
"\n",
"def encode(word):\n",
" index = words.index(word)\n",
" binary = [0] * len(words)\n",
" binary[index] = 1\n",
" return binary\n",
"\n",
"def decode(pattern):\n",
" winner = max(pattern)\n",
" index = list(pattern).index(winner)\n",
" return label(index)\n",
"\n",
"def label(index):\n",
" for word, pattern in patterns.items():\n",
" if pattern[index] == 1:\n",
" return word\n",
" return None\n",
"\n",
"\n",
"# Reset to max length:\n",
"pattern_size = len(encode(words[0]))\n",
"\n",
"patterns = {word: encode(word) for word in words}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"text_words is the text corpus, as a list of words. Your's will be too long to display here."
]
},
{
"cell_type": "code",
"execution_count": 220,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"['me',\n",
" 'like',\n",
" 'you',\n",
" 'you',\n",
" 'like',\n",
" 'me',\n",
" 'me',\n",
" 'like',\n",
" 'apples',\n",
" 'me',\n",
" 'like',\n",
" 'bananas',\n",
" 'you',\n",
" 'like',\n",
" 'bananas',\n",
" 'you',\n",
" 'like',\n",
" 'apples',\n",
" 'you',\n",
" 'hate',\n",
" 'berries',\n",
" 'me',\n",
" 'like',\n",
" 'berries',\n",
" 'me',\n",
" 'need',\n",
" 'berries',\n",
" 'you',\n",
" 'need',\n",
" 'apples',\n",
" 'you',\n",
" 'need',\n",
" 'me']"
]
},
"execution_count": 220,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"text_words"
]
},
{
"cell_type": "code",
"execution_count": 221,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"['berries', 'need', 'me', 'apples', 'like', 'bananas', 'you', 'hate']"
]
},
"execution_count": 221,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"words"
]
},
{
"cell_type": "code",
"execution_count": 222,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"dict_keys(['berries', 'bananas', 'need', 'you', 'me', 'like', 'hate', 'apples'])"
]
},
"execution_count": 222,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"patterns.keys()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Testing our encoding and decoding functions:"
]
},
{
"cell_type": "code",
"execution_count": 223,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[0, 1, 0, 0, 0, 0, 0, 0]"
]
},
"execution_count": 223,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"encode(\"need\")"
]
},
{
"cell_type": "code",
"execution_count": 224,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"'need'"
]
},
"execution_count": 224,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"decode(encode(\"need\"))"
]
},
{
"cell_type": "code",
"execution_count": 247,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"'need'"
]
},
"execution_count": 247,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"decode([0, 0.6, 0.5, 0, 0.1, 0, 0, 0])"
]
},
{
"cell_type": "code",
"execution_count": 229,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"'need'"
]
},
"execution_count": 229,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"label(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And now, we explore reading through a text, predicting what word comes next."
]
},
{
"cell_type": "code",
"execution_count": 230,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from conx import SRN\n",
"\n",
"class Predict(SRN):\n",
" def initialize_inputs(self):\n",
" pass\n",
"\n",
" def inputs_size(self):\n",
" # Return the number of inputs:\n",
" return len(text_words)\n",
"\n",
" def get_inputs(self, i):\n",
" current_word = text_words[i]\n",
" next_word = text_words[(i + 1) % len(text_words)]\n",
" return (patterns[current_word], patterns[next_word])"
]
},
{
"cell_type": "code",
"execution_count": 231,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"net = Predict(len(encode(\"need\")), 5, len(encode(\"need\")))"
]
},
{
"cell_type": "code",
"execution_count": 232,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------\n",
"Training for max trails: 2000 ...\n",
"Epoch: 0 TSS error: 101.397531278 %correct: 0.0\n",
"Epoch: 100 TSS error: 21.0288009846 %correct: 0.0\n",
"Epoch: 200 TSS error: 17.6554494885 %correct: 0.18181818181818182\n",
"Epoch: 300 TSS error: 14.6845872245 %correct: 0.30303030303030304\n",
"Epoch: 400 TSS error: 12.045306326 %correct: 0.36363636363636365\n",
"Epoch: 500 TSS error: 10.2342781218 %correct: 0.3939393939393939\n",
"Epoch: 600 TSS error: 9.24119739812 %correct: 0.48484848484848486\n",
"Epoch: 700 TSS error: 8.25721892617 %correct: 0.5151515151515151\n",
"Epoch: 800 TSS error: 7.19393364174 %correct: 0.5454545454545454\n",
"Epoch: 900 TSS error: 6.36376877071 %correct: 0.6363636363636364\n",
"Epoch: 1000 TSS error: 5.89511765761 %correct: 0.696969696969697\n",
"Epoch: 1100 TSS error: 5.55586028248 %correct: 0.696969696969697\n",
"Epoch: 1200 TSS error: 4.87985305528 %correct: 0.696969696969697\n",
"Epoch: 1300 TSS error: 4.19677499327 %correct: 0.7272727272727273\n",
"Epoch: 1400 TSS error: 3.75729352055 %correct: 0.7878787878787878\n",
"Epoch: 1500 TSS error: 3.51941705011 %correct: 0.7878787878787878\n",
"Epoch: 1600 TSS error: 3.74943253468 %correct: 0.7272727272727273\n",
"Epoch: 1700 TSS error: 4.20421350651 %correct: 0.7575757575757576\n",
"Epoch: 1800 TSS error: 3.84047673151 %correct: 0.696969696969697\n",
"Epoch: 1900 TSS error: 5.23020456274 %correct: 0.6666666666666666\n",
"Epoch: 2000 TSS error: 8.41181206316 %correct: 0.5757575757575758\n",
"--------------------------------------------------\n",
"Epoch: 2000 TSS error: 8.41181206316 %correct: 0.5757575757575758\n"
]
}
],
"source": [
"net.train(max_training_epochs=2000,\n",
" report_rate=100,\n",
" tolerance=0.3,\n",
" epsilon=0.1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And testing the trained network. You may have to train an amount comparable to what Elman did."
]
},
{
"cell_type": "code",
"execution_count": 233,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------\n",
"Test:\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 2.11047114e-02 1.05353114e-03 1.51418711e-02 1.19526865e-05\n",
" 8.90758773e-01 1.76233384e-04 2.73609951e-03 9.97449915e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 1.68494595e-01 5.12335672e-07 2.80982943e-03 3.66894766e-01\n",
" 5.04145564e-02 1.01438571e-01 3.87937518e-01 2.34307921e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 2.49153116e-03 4.51391506e-06 6.51019511e-04 2.87403159e-05\n",
" 2.20248631e-02 1.04906100e-04 5.16446136e-01 1.33050121e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 7.63738588e-03 7.70121481e-05 5.42627790e-05 1.82421577e-02\n",
" 9.99785179e-01 3.71534302e-06 3.81533227e-03 7.47507437e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 2.72138737e-03 1.03817769e-06 6.68314947e-01 1.97579083e-02\n",
" 2.29589172e-03 3.11517701e-01 2.24290093e-02 9.05405235e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 2.43303730e-04 9.62138636e-04 7.49287384e-01 7.96864676e-05\n",
" 4.42785887e-01 4.81448906e-05 8.76438154e-04 1.76318817e-05] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 2.45163579e-03 1.00933830e-04 7.95025710e-02 2.01677608e-05\n",
" 7.58015260e-01 1.26940071e-03 2.15958286e-03 1.05626171e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 3.01853069e-01 7.01521263e-07 4.03404015e-03 1.44094632e-01\n",
" 5.55887688e-02 8.51190761e-02 2.98152682e-01 2.33844260e-03] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 1, 0, 0, 0, 0]\n",
"Output: [ 1.36616366e-03 3.71843192e-05 7.78310091e-01 3.21284946e-04\n",
" 4.74596843e-03 1.47235183e-03 1.23189297e-02 1.82179371e-05] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 1.02372008e-04 1.38967935e-02 4.76467481e-02 1.19610946e-03\n",
" 9.81004295e-01 2.58341082e-06 7.96482584e-04 6.69151434e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 2.86566945e-01 1.98456049e-07 4.45602436e-02 2.30433682e-02\n",
" 3.02662121e-03 7.22371093e-01 2.05122679e-01 1.91805679e-04] \n",
"Target: [0, 0, 0, 0, 0, 1, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 1, 0, 0]\n",
"Output: [ 9.41504760e-03 1.83388393e-06 9.65616367e-05 2.15182954e-04\n",
" 3.36982844e-04 1.28766213e-04 9.83429747e-01 2.18583078e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 7.97012470e-06 1.06170275e-03 1.58996553e-05 4.68757002e-03\n",
" 9.94052621e-01 1.68900137e-07 2.45129043e-04 1.40333322e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 1.50275642e-04 5.41158091e-06 4.49034615e-01 3.04748027e-01\n",
" 1.31470267e-02 2.20919895e-02 1.71914396e-02 1.93466850e-05] \n",
"Target: [0, 0, 0, 0, 0, 1, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 1, 0, 0]\n",
"Output: [ 9.37406148e-02 8.82288042e-04 6.42291232e-05 2.32777862e-04\n",
" 4.43631892e-06 4.87642341e-03 9.90319422e-01 2.55908234e-02] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 1.83441629e-06 2.32226398e-02 8.77017301e-07 6.04655782e-03\n",
" 6.69673881e-01 2.08785137e-07 5.71574746e-03 6.48453780e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 4.52176240e-05 4.20894132e-05 5.19098230e-01 6.27089146e-01\n",
" 9.53376179e-02 4.79575818e-04 5.24011345e-03 3.22883694e-05] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 1, 0, 0, 0, 0]\n",
"Output: [ 2.65817159e-02 4.63910461e-02 3.30035595e-03 8.91241319e-04\n",
" 6.64390206e-07 4.08686769e-02 9.87687801e-01 5.37722203e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 1.68474568e-05 4.06370963e-01 4.88651551e-08 2.77661775e-02\n",
" 2.36846565e-01 1.70497547e-06 5.35921607e-02 5.27398170e-01] \n",
"Target: [0, 0, 0, 0, 0, 0, 0, 1] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 0, 1]\n",
"Output: [ 9.88908880e-01 7.74060325e-07 5.21202381e-03 1.01424465e-02\n",
" 1.05953996e-01 8.63489687e-03 1.20903664e-02 4.34977768e-04] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [1, 0, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 1.36752918e-03 4.77953113e-06 9.97719464e-01 3.31942383e-04\n",
" 6.71359503e-03 2.37344984e-03 3.76397287e-02 1.32550461e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 1.51596227e-03 7.87156987e-03 8.53569453e-02 2.54742654e-05\n",
" 7.20870026e-01 6.18437659e-05 7.34336944e-04 2.75104430e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 1, 0, 0, 0]\n",
"Output: [ 3.44544470e-01 7.04084292e-07 6.45833494e-03 8.13592158e-02\n",
" 1.95728602e-02 1.16880265e-01 2.97796244e-01 1.55181788e-03] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [1, 0, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 3.17410440e-04 3.14687237e-05 9.90580751e-01 3.37329048e-04\n",
" 1.80777849e-03 1.14119762e-03 1.72421593e-03 1.53216105e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 1.32958219e-04 8.20324789e-01 3.19829350e-02 1.11927946e-03\n",
" 9.66358774e-02 6.74801482e-05 4.44381739e-03 1.56713854e-03] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [0, 1, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 9.87732840e-01 5.43951693e-06 1.78157917e-04 5.90880053e-02\n",
" 5.17530874e-05 3.13743263e-03 1.93372204e-01 5.27002538e-04] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input : [1, 0, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 9.92419619e-06 2.41993098e-03 5.75067467e-02 8.12808783e-02\n",
" 1.65370076e-04 1.33069140e-05 7.71008160e-01 2.78237587e-04] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 3.28256519e-05 4.35508073e-01 1.10829858e-05 2.08800174e-03\n",
" 1.08769399e-03 9.32113337e-05 9.77990744e-02 4.88744134e-02] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 1, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 2.15241661e-03 3.47028193e-03 8.16172703e-02 6.80181995e-01\n",
" 7.56172012e-02 1.49831933e-05 1.66703890e-03 1.58680040e-04] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 0, 1, 0, 0, 0, 0]\n",
"Output: [ 3.08310623e-02 2.05344187e-02 5.10135153e-02 4.00659795e-03\n",
" 1.52332171e-06 1.09967567e-01 9.87389738e-01 2.50205606e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input : [0, 0, 0, 0, 0, 0, 1, 0]\n",
"Output: [ 2.34789454e-04 4.59475180e-02 4.95691855e-09 2.00150300e-02\n",
" 5.22988282e-01 1.82713086e-06 1.74963293e-01 8.22480171e-01] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 1, 0, 0, 0, 0, 0, 0]\n",
"Output: [ 2.37807731e-04 4.32556902e-05 4.74251992e-01 1.41518631e-01\n",
" 5.05144874e-02 2.25766644e-04 3.15199893e-04 9.34517546e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input : [0, 0, 1, 0, 0, 0, 0, 0]\n",
"Output: [ 8.28121014e-04 3.47704943e-04 9.87438973e-01 6.39413460e-05\n",
" 4.71882961e-02 9.89063295e-04 5.96253336e-04 4.24409629e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"--------------------------------------------------\n",
"Epoch: 2000 TSS error: 44.6544609703 %correct: 0.5757575757575758\n"
]
}
],
"source": [
"net.test()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That is hard to read. conx comes with a way to override the display of the test input:"
]
},
{
"cell_type": "code",
"execution_count": 234,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"net.display_test_input = lambda inputs: print(\"Input: \" + decode(inputs))"
]
},
{
"cell_type": "code",
"execution_count": 235,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------\n",
"Test:\n",
"******************************\n",
"Input: me\n",
"Output: [ 2.19249592e-02 1.17853572e-03 1.51508181e-02 1.17684923e-05\n",
" 8.80658274e-01 1.78081640e-04 2.75980500e-03 1.02082983e-03] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 1.68633200e-01 5.37501743e-07 2.67303674e-03 3.83480112e-01\n",
" 5.36311967e-02 9.12961568e-02 3.90337956e-01 2.49953788e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Incorrect\n",
"******************************\n",
"Input: you\n",
"Output: [ 2.16347056e-03 4.82952415e-06 8.08335137e-04 2.89122275e-05\n",
" 2.06314123e-02 1.06524202e-04 4.95778876e-01 1.15916079e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Incorrect\n",
"******************************\n",
"Input: you\n",
"Output: [ 9.61742864e-03 6.69739761e-05 4.38371364e-05 1.83955093e-02\n",
" 9.99758159e-01 3.91596066e-06 5.33965782e-03 8.75034860e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 1.59776461e-03 1.29878715e-06 7.29287646e-01 2.11596787e-02\n",
" 2.29595164e-03 2.55048556e-01 1.74241902e-02 6.95112533e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: me\n",
"Output: [ 3.07335424e-04 1.39058538e-03 7.41881211e-01 5.20402761e-05\n",
" 3.72726884e-01 6.48180062e-05 7.46687040e-04 1.89878503e-05] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: me\n",
"Output: [ 3.87894545e-03 1.10862830e-04 7.04280442e-02 1.92538352e-05\n",
" 8.21127263e-01 9.34128216e-04 2.25373282e-03 1.48188002e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 2.56192982e-01 5.97268959e-07 3.62678330e-03 1.70199664e-01\n",
" 4.94225080e-02 1.08877521e-01 3.20287196e-01 2.14924752e-03] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: apples\n",
"Output: [ 1.40579901e-03 3.75261233e-05 7.76999273e-01 3.16727630e-04\n",
" 5.09474389e-03 1.45644894e-03 1.03626231e-02 1.76586519e-05] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: me\n",
"Output: [ 1.06682722e-04 1.23240065e-02 4.99856722e-02 1.10853709e-03\n",
" 9.81406606e-01 2.62842450e-06 7.84537135e-04 6.36615205e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 2.85432805e-01 1.99667648e-07 4.40357606e-02 2.27686046e-02\n",
" 3.04919084e-03 7.22351492e-01 2.06156247e-01 1.94165162e-04] \n",
"Target: [0, 0, 0, 0, 0, 1, 0, 0] Correct\n",
"******************************\n",
"Input: bananas\n",
"Output: [ 9.36015374e-03 1.82969909e-06 9.69154178e-05 2.15335855e-04\n",
" 3.40391797e-04 1.28283504e-04 9.83131423e-01 2.17614662e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input: you\n",
"Output: [ 7.98351243e-06 1.06036439e-03 1.58897282e-05 4.68884010e-03\n",
" 9.94042858e-01 1.69013953e-07 2.45893941e-04 1.40448823e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 1.49736082e-04 5.42130522e-06 4.49134660e-01 3.05200444e-01\n",
" 1.31620030e-02 2.20300271e-02 1.71746246e-02 1.93392022e-05] \n",
"Target: [0, 0, 0, 0, 0, 1, 0, 0] Incorrect\n",
"******************************\n",
"Input: bananas\n",
"Output: [ 9.36684760e-02 8.84306483e-04 6.43202615e-05 2.32915976e-04\n",
" 4.42099127e-06 4.88098069e-03 9.90338740e-01 2.55761337e-02] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input: you\n",
"Output: [ 1.83353213e-06 2.33225277e-02 8.76377696e-07 6.05267999e-03\n",
" 6.69178789e-01 2.09102421e-07 5.72078856e-03 6.49224323e-02] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: like\n",
"Output: [ 4.53289548e-05 4.21343554e-05 5.19203271e-01 6.26932465e-01\n",
" 9.54529086e-02 4.78352218e-04 5.23400907e-03 3.23054064e-05] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: apples\n",
"Output: [ 2.65812939e-02 4.63874061e-02 3.29794795e-03 8.91123398e-04\n",
" 6.64966748e-07 4.08808152e-02 9.87676790e-01 5.37781493e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input: you\n",
"Output: [ 1.68605982e-05 4.06146673e-01 4.88406973e-08 2.77651713e-02\n",
" 2.36912270e-01 1.70499941e-06 5.36105997e-02 5.27476511e-01] \n",
"Target: [0, 0, 0, 0, 0, 0, 0, 1] Incorrect\n",
"******************************\n",
"Input: hate\n",
"Output: [ 9.88890843e-01 7.74040903e-07 5.21700364e-03 1.01339593e-02\n",
" 1.05964210e-01 8.64054159e-03 1.20940010e-02 4.35022912e-04] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: berries\n",
"Output: [ 1.36747808e-03 4.77960702e-06 9.97719282e-01 3.31952967e-04\n",
" 6.71283039e-03 2.37359387e-03 3.76371420e-02 1.32549355e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: me\n",
"Output: [ 1.51576315e-03 7.87347276e-03 8.53596693e-02 2.54780360e-05\n",
" 7.20835826e-01 6.18437867e-05 7.34368984e-04 2.75107671e-04] \n",
"Target: [0, 0, 0, 0, 1, 0, 0, 0] Correct\n",
"******************************\n",
"Input: like\n",
"Output: [ 3.44568631e-01 7.04157396e-07 6.45846718e-03 8.13533260e-02\n",
" 1.95734872e-02 1.16863140e-01 2.97789985e-01 1.55190772e-03] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: berries\n",
"Output: [ 3.17413297e-04 3.14686255e-05 9.90580720e-01 3.37330770e-04\n",
" 1.80775522e-03 1.14119261e-03 1.72434594e-03 1.53221006e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: me\n",
"Output: [ 1.32954070e-04 8.20331629e-01 3.19825450e-02 1.11932636e-03\n",
" 9.66358241e-02 6.74787980e-05 4.44383620e-03 1.56716210e-03] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: need\n",
"Output: [ 9.87733221e-01 5.43966541e-06 1.78155206e-04 5.90902170e-02\n",
" 5.17517026e-05 3.13741905e-03 1.93369006e-01 5.26993932e-04] \n",
"Target: [1, 0, 0, 0, 0, 0, 0, 0] Correct\n",
"******************************\n",
"Input: berries\n",
"Output: [ 9.92407044e-06 2.42001866e-03 5.75066338e-02 8.12811261e-02\n",
" 1.65360061e-04 1.33070947e-05 7.71018275e-01 2.78239813e-04] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input: you\n",
"Output: [ 3.28237683e-05 4.35526262e-01 1.10820617e-05 2.08812196e-03\n",
" 1.08775153e-03 9.32060592e-05 9.77945742e-02 4.88765141e-02] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: need\n",
"Output: [ 2.15271195e-03 3.47011746e-03 8.16155037e-02 6.80198988e-01\n",
" 7.56157304e-02 1.49827629e-05 1.66695218e-03 1.58669707e-04] \n",
"Target: [0, 0, 0, 1, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: apples\n",
"Output: [ 3.08317786e-02 2.05337084e-02 5.10131164e-02 4.00662821e-03\n",
" 1.52320789e-06 1.09967841e-01 9.87391212e-01 2.50207820e-03] \n",
"Target: [0, 0, 0, 0, 0, 0, 1, 0] Correct\n",
"******************************\n",
"Input: you\n",
"Output: [ 2.34761297e-04 4.59474863e-02 4.95743532e-09 2.00144274e-02\n",
" 5.22990532e-01 1.82703964e-06 1.74951905e-01 8.22466840e-01] \n",
"Target: [0, 1, 0, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: need\n",
"Output: [ 2.37808011e-04 4.32540607e-05 4.74280040e-01 1.41513223e-01\n",
" 5.05119183e-02 2.25784525e-04 3.15222967e-04 9.34475450e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Incorrect\n",
"******************************\n",
"Input: me\n",
"Output: [ 8.28123060e-04 3.47750600e-04 9.87437099e-01 6.39344593e-05\n",
" 4.71903866e-02 9.88990231e-04 5.96227738e-04 4.24438759e-06] \n",
"Target: [0, 0, 1, 0, 0, 0, 0, 0] Correct\n",
"--------------------------------------------------\n",
"Epoch: 2000 TSS error: 44.7775832351 %correct: 0.6060606060606061\n"
]
}
],
"source": [
"net.test()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That is better. But we can also do the same for displaying the outputs:"
]
},
{
"cell_type": "code",
"execution_count": 236,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def display_outputs(outputs, result=\"Outputs\", label=None): \n",
" print(result + \": \" + decode(outputs))"
]
},
{
"cell_type": "code",
"execution_count": 237,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"net.display_test_output = display_outputs"
]
},
{
"cell_type": "code",
"execution_count": 238,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------\n",
"Test:\n",
"******************************\n",
"Input: me\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: you\n",
"Incorrect: you\n",
"******************************\n",
"Input: you\n",
"Outputs: you\n",
"Incorrect: you\n",
"******************************\n",
"Input: you\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: me\n",
"Correct: me\n",
"******************************\n",
"Input: me\n",
"Outputs: me\n",
"Incorrect: me\n",
"******************************\n",
"Input: me\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: you\n",
"Incorrect: apples\n",
"******************************\n",
"Input: apples\n",
"Outputs: me\n",
"Correct: me\n",
"******************************\n",
"Input: me\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: bananas\n",
"Correct: bananas\n",
"******************************\n",
"Input: bananas\n",
"Outputs: you\n",
"Correct: you\n",
"******************************\n",
"Input: you\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: me\n",
"Incorrect: bananas\n",
"******************************\n",
"Input: bananas\n",
"Outputs: you\n",
"Correct: you\n",
"******************************\n",
"Input: you\n",
"Outputs: like\n",
"Incorrect: like\n",
"******************************\n",
"Input: like\n",
"Outputs: apples\n",
"Incorrect: apples\n",
"******************************\n",
"Input: apples\n",
"Outputs: you\n",
"Correct: you\n",
"******************************\n",
"Input: you\n",
"Outputs: hate\n",
"Incorrect: hate\n",
"******************************\n",
"Input: hate\n",
"Outputs: berries\n",
"Correct: berries\n",
"******************************\n",
"Input: berries\n",
"Outputs: me\n",
"Correct: me\n",
"******************************\n",
"Input: me\n",
"Outputs: like\n",
"Correct: like\n",
"******************************\n",
"Input: like\n",
"Outputs: berries\n",
"Incorrect: berries\n",
"******************************\n",
"Input: berries\n",
"Outputs: me\n",
"Correct: me\n",
"******************************\n",
"Input: me\n",
"Outputs: need\n",
"Correct: need\n",
"******************************\n",
"Input: need\n",
"Outputs: berries\n",
"Correct: berries\n",
"******************************\n",
"Input: berries\n",
"Outputs: you\n",
"Correct: you\n",
"******************************\n",
"Input: you\n",
"Outputs: need\n",
"Incorrect: need\n",
"******************************\n",
"Input: need\n",
"Outputs: apples\n",
"Incorrect: apples\n",
"******************************\n",
"Input: apples\n",
"Outputs: you\n",
"Correct: you\n",
"******************************\n",
"Input: you\n",
"Outputs: hate\n",
"Incorrect: need\n",
"******************************\n",
"Input: need\n",
"Outputs: me\n",
"Incorrect: me\n",
"******************************\n",
"Input: me\n",
"Outputs: me\n",
"Correct: me\n",
"--------------------------------------------------\n",
"Epoch: 2000 TSS error: 44.777553161 %correct: 0.6060606060606061\n"
]
}
],
"source": [
"net.test()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Better! Why is it that sometimes the \"output\" may be the same as \"correct\" but still marked as \"Incorrect\"?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2. Analysis"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Elman produced \"dendograms\" (tree plots) to show similarity of the hidden activations associated with each word. We can do the same."
]
},
{
"cell_type": "code",
"execution_count": 239,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import io\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from scipy.cluster import hierarchy\n",
"from scipy.spatial import distance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will plot the hidden unit activations of how close each hidden pattern is to each other. This is a way of seeing the clustering among numeric representations of many dimensions. \n",
"\n",
"To do this, we need to get the \"hidden layer activations\" for each word, in a proper order."
]
},
{
"cell_type": "code",
"execution_count": 240,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([ 0.94643207, 0.02253122, 0.75723384, 0.03539211, 0.99686973])"
]
},
"execution_count": 240,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"net.layer[0].propagate(encode(\"need\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we will go through the words in the text, and get the hidden layer activations.\n",
"\n",
"Note that each time, we are overwriting the previous activations. A better method would be to somehow average each set of hidden layer activations."
]
},
{
"cell_type": "code",
"execution_count": 241,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"hiddens_dict = {}\n",
"for word in text_words:\n",
" hiddens_dict[word] = net.layer[0].propagate(patterns[word])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we get those hidden layer activations in the order that matches the \"words\" list:"
]
},
{
"cell_type": "code",
"execution_count": 242,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"hiddens = []\n",
"for word in words:\n",
" hiddens.append(hiddens_dict[word])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we are ready to process the hidden layer activations to display as a dendrogram. \n",
"\n",
"http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.dendrogram.html"
]
},
{
"cell_type": "code",
"execution_count": 243,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"linkage = hierarchy.linkage(hiddens)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's make the output big enough to easily see (units are based on DPI):"
]
},
{
"cell_type": "code",
"execution_count": 244,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plt.rcParams[\"figure.figsize\"] = (13, 5)"
]
},
{
"cell_type": "code",
"execution_count": 245,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"threshold = 0.3\n",
"clusters = hierarchy.fcluster(linkage, threshold, criterion=\"distance\")"
]
},
{
"cell_type": "code",
"execution_count": 246,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 246,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAwYAAAFsCAYAAAB2Nq7BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYZHV97/H3h2GRVWYASRgcXKISibhFgnsrUdCoGG9c\niCFo9MqTGzVqjEhy1dGYmHhN3CBRFBcwBvcEXLnX2KBBRBNxQTY39qA4jSgojuP3/nGqh6Knu6dk\n6tSZOvN+Pc88U3XqdPV3fs+Z6v6c35aqQpIkSdK2bbuuC5AkSZLUPYOBJEmSJIOBJEmSJIOBJEmS\nJAwGkiRJkjAYSJIkSQK277qAUSVxXVVJkiRpDKoqC4+1GgySnAw8Dri2qg5e5PXfB44bPP0R8MdV\n9bWl3s89FyRJkqQtk2ySCYD2hxK9Ezh8mde/DTysqu4NvBp4W8v1SJIkSVpEqz0GVfW5JAcs8/q5\nQ0/PBVa3WY8kSZKkxW1Nk4+fDXyi6yIkSZKkbdFWMfk4ySOAZwIP6boWSZIkaVvUeTBIcjBwEnBE\nVc0td+7atWs3Pp6ZmWFmZqbV2iRJkqRpNzs7y+zs7GbPS9sr/SS5E3BGVd1rkdfWAJ8Gjl4w32Cx\n9ylXJZIkSZK2TJJFlyttNRgkeS8wA+wFXAu8AtgRqKo6KcnbgCcBlwEB1lfVIUu8l8FAkiRJ2kKd\nBINxMhhIkiRJW26pYLA1rUokSZIkqSMGA0mSJEkGA0mSJElbwXKlfbdqFcwtuwirJI1u5UpYt67r\nKiRJfeTk45YlMIVlS9pK+ZkiSdpSTj6WJEmStCSDgSRJkiSDgSRJkiSDgSRJkiQMBpIkSZIwGEiS\nJEnCYCBJkiQJg4EkSZIkDAaSJEmSMBhIkiRJwmAgSZIkCYOBJEmSJAwGkiRJkjAYSJIkScJgIEmS\nJAmDgSRJkiQMBpIkSZKA7bsuQNqWrVoFc3NdV6Fpk3RdgabNypWwbl3XVUja2qWquq5hJElqWmod\nlsAUlq0J8fqQNAl+1kgaloSq2uQ2k0OJJEmSJBkMJEmSJBkMJEmSJGEwkCRJkoTBQJIkSRIGA0mS\nJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwkSZIkYTCQJEmShMFAkiRJEi0HgyQnJ7k2yVeX\nOedNSS5Ncn6S+7RZjyRJkqTFtd1j8E7g8KVeTPIY4K5VdTfgWOAtLdcjSZIkaRGtBoOq+hwwt8wp\nRwKnDM79AnD7JPu2WZMkSZKkTXU9x2A1cMXQ86sGxyRJkiRN0PZdF/DLWLt27cbHMzMzzMzMdFaL\nJEmSNA1mZ2eZnZ3d7HmpqlYLSXIAcEZVHbzIa28BPlNV7xs8vwh4eFVdu8i51XatbUhgCsvWhHh9\nSJoEP2skDUtCVWXh8UkMJcrgz2JOB/4QIMmhwPWLhQJJkiRJ7Wp1KFGS9wIzwF5JLgdeAewIVFWd\nVFUfT/LYJN8EbgSe2WY9kiRJkhbX+lCicXEokfrI60PSJPhZI2lYl0OJJEmSJG3lDAaSJEmSDAaS\nJEmSDAaSJEmSMBhIkiRJwmAgSZIkCYOBJEmSJAwGkiRJkjAYSJIkScJgIEmSJAmDgSRJkiQMBpIk\nSZIwGEiSJEnCYCBJkiQJ2L7rAiRJ/bFqFczNdV2FFpN0XYEWWrkS1q3rugrpFqmqrmsYSZKallqH\nJTCFZWtCvD7UN17T0uj8/6KuJKGqNrld4FAiSZIkSQYDSZIkSQYDSZIkSRgMJEmSJGEwkCRJkoTB\nQJIkSRIGA0mSJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwkSZIkYTCQJEmShMFAkiRJEgYD\nSZIkSRgMJEmSJGEwkCRJkoTBQJIkSRIGA0mSJEkYDCRJkiQxgWCQ5IgkFyW5JMlxi7y+R5LTk5yf\n5GtJntF2TZIkSZJuLVXV3psn2wGXAIcBVwNfBJ5WVRcNnXM8sEdVHZ9kb+BiYN+q+vmC96o2a21L\nAlNYtibE60N94zUtjc7/L+pKEqoqC49v3/L3PQS4tKouGxRxGnAkcNHQOQXsPni8O/CDhaFAkiRN\nv1WrYG6u6yq2LtnkV7Nt18qVsG5d11Vs29oOBquBK4aeX0kTFoadAJye5GpgN+CpLdckSZI6MDfn\nHXItzZDUva1h8vHhwJeraj/gvsCJSXbruCZJkiRpm9J2j8FVwJqh5/sPjg17JvAagKr6VpLvAAcC\nX1r4ZmvXrt34eGZmhpmZmfFWK0mSJPXM7Owss7Ozmz2v7cnHK2gmEx8GXAOcBxxVVRcOnXMi8L2q\nemWSfWkCwb2rat2C93LysXrH60N94zWt5Xh9aDleH5PTyeTjqtqQ5LnAmTTDlk6uqguTHNu8XCcB\nrwbeleSrgy97ycJQIEmSJKldrfYYjJM9Buojrw/1jde0luP1oeV4fUzOUj0GW8PkY0mSJEkdMxhI\nkiRJMhhIkiRJMhhIkiRJov19DCRJkjRhq1Y1O01Pm2nb/XjlSljXo7U0DQaSJEk9MzfnCj+TMG1B\nZnMcSiRJkiTJYCBJkiTJYCBJkiQJg4EkSZIkDAaSJEmSMBhIkiRJwmAgSZIkCYOBJEmSJAwGkiRJ\nkjAYSJIkScJgIEmSJAmDgSRJkiQMBpIkSZL4JYJBkgOS/Pbg8c5Jdm+vLEmSJEmTNFIwSPI/gQ8C\nbx0c2h/417aKkiRJkjRZo/YY/AnwYOAGgKq6FLhDW0VJkiRJmqxRg8HNVfWz+SdJtgeqnZIkSZIk\nTdqoweCsJH8B7JzkUcAHgDPaK0uSJEnSJKVq8zf+k2wHPAt4NBDgU8Dba5QvHpMkk/x2Y5PAFJat\nCfH6UN94TWs5Xh+TY1tPxrS2cxKqKpscHzEY7Ar8tKo2DJ6vAHaqqpvGXunSNRgM1DteH+obr2kt\nx+tjcmzryZjWdl4qGIw6lOjTwM5Dz3cG/t84CpMkSZLUvVGDwe2q6sfzTwaPd2mnJEmSJEmTNmow\nuDHJ/eafJLk/8JN2SpIkSZI0aduPeN4LgA8kuZpm8vGvAE9trSpJkiRJEzXS5GOAJDsA9xg8vbiq\n1rdW1eLf38nH6h2vD/WN17SW4/UxObb1ZExrO2/RqkSDN3gQcCeGehmq6pRxFTjC9zcYqHe8PtQ3\nXtNajtfH5NjWkzGt7bxUMBhpKFGSU4G7AucDGwaHC5hYMJAkSZLUnlHnGPwmcM+pvGUvSZIkabNG\nXZXo6zQTjiVJkiT10Kg9BnsD30hyHnDz/MGqekIrVUmSJEmaqFGDwdo2i5AkSZLUrZFXJbrN3yA5\nAngDzbClk6vq7xY5ZwZ4PbAD8P2qesQi50zlFIdpna2uyfD6UN94TWs5Xh+TY1tPxrS28xYtV5rk\nUODNwK8DOwIrgBurao/NfN12wCXAYcDVwBeBp1XVRUPn3B44B3h0VV2VZO+qum6R9zIYqHe8PtQ3\nXtNajtfH5NjWkzGt7bxUMBh18vEJwFHApcDOwLOBE0f4ukOAS6vqssGGaKcBRy445/eBD1XVVQCL\nhQJJkiRJ7Ro1GFBV3wRWVNWGqnoncMQIX7YauGLo+ZWDY8PuDqxK8pkkX0xy9Kg1SZIkSRqPUScf\n35RkR+D8JK8FruGXCBUj1HA/4JHArsDnk3x+EERuZe3atRsfz8zMMDMzM6YSJEmSpH6anZ1ldnZ2\ns+eNOsfgAOBamvkFLwRuD5xYVd/azNcdCqytqiMGz18K1PAE5CTHAberqlcOnr8d+ERVfWjBeznH\nQL3j9aG+8ZrWcrw+Jse2noxpbectnWPwxKr6aVXdUFWvrKoXAY8b4eu+CPxakgMGPQ5PA05fcM6/\nAQ9JsiLJLsBvAReOWJckSZKkMRg1GByzyLFnbO6LqmoD8FzgTOAC4LSqujDJsUmeMzjnIuBTwFeB\nc4GTquobI9YlSZIkaQyWHUqU5CiaVYMeAnx26KU9gA1VdVi75d2qFocSqXe8PtQ3XtNajtfH5NjW\nkzGt7bzUUKLNTT4+h2ai8d7A3w8d/xHNHX5pq7BqFczNdV3FbZNN/ltu3VauhHXruq5CkiSN26iT\nj3cFflJVv0hyd+BAmgnC69sucKgGewy0JNt5cmxrLcfrQ8vx+pgc23oyprWdt3Ty8dnA7ZKsppkv\ncDTwrvGVJ0mSJKlLowaDVNVNwJOAf6yqJwMHtVeWJEmSpEkaORgkeSDwdOBjg2Mr2ilJkiRJ0qSN\nGgxeABwPfKSqLkhyF+Az7ZUlSZIkaZJGmny8NXDysZZjO0+Oba3leH1oOV4fk2NbT8a0tvNtWq40\nyRuq6gVJzgA2+WdX1RPGWKMkSZKkjmxuH4NTB3+/ru1CJEmSJHVn5KFESfYBqKrvt1rR0t/foURa\nku08Oba1luP1oeV4fUyObT0Z09rOt3kfgyRrk1wHXAxckuT7SV7eRpGSJEmSurFsMEjyIuDBwAOq\nalVVrQR+C3hwkhdOokBJkiRJ7Vt2KFGSLwOPqqrrFhzfBzizqu7bcn3D39OhRFqS7Tw5trWW4/Wh\n5Xh9TI5tPRnT2s63dSjRDgtDAWycZ7DDuIqTJEmS1K3NBYOf3cbXJEmSJE2RzS1Xeu8kNyxyPMDt\nWqhHkiRJUgeWDQZVtWJShUiSJEnqzmaXK5UkSZLUfwYDSZIkSQYDSZIkSQYDSZIkSRgMJEmSJGEw\nkCRJkoTBQJIkSRIGA0mSJEkYDCRJkiSxmZ2PJUndWbUK5ua6ruKXl3RdwS9n5UpYt67rKiSpewYD\nSdpKzc1BVddV9N+0BRlJaotDiSRJkiQZDCRJkiQZDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwk\nSZIkYTCQJEmSxASCQZIjklyU5JIkxy1z3gOSrE/ypLZrkiRJknRrrQaDJNsBJwCHAwcBRyU5cInz\n/hb4VJv1SJIkSVpc2z0GhwCXVtVlVbUeOA04cpHzngd8EPhey/VIkiRJWkTbwWA1cMXQ8ysHxzZK\nsh/wxKr6JyAt1yNJkiRpEVvD5OM3AMNzDwwHkiRJ0oRt3/L7XwWsGXq+/+DYsN8ETksSYG/gMUnW\nV9XpC99s7dq1Gx/PzMwwMzMz7nolSZKkXpmdnWV2dnaz56WqWisiyQrgYuAw4BrgPOCoqrpwifPf\nCZxRVR9e5LVqs9a2JDCFZU8d23lybOvJsa0nw3aeHNt6cmzryZjWdk5CVW0ySqfVHoOq2pDkucCZ\nNMOWTq6qC5Mc27xcJy38kjbrkSRJkrS4VnsMxskeAy3Hdp4c23pybOvJsJ0nx7aeHNt6Mqa1nZfq\nMdgaJh9LkiRJ6pjBQJIkSZLBQJIkSZLBQJIkSRIGA0mSJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJ\nkiRJGAwkSZIkYTCQJEmShMFAkiRJEgYDSZIkSRgMJEmSJGEwkCRJkoTBQJIkSRIGA0mSJEkYDCRJ\nkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwkSZIkYTCQJEmShMFAkiRJEgYDSZIkSRgMJEmSJGEwkCRJ\nkoTBQJIkSRIGA0mSJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwkSZIkYTCQJEmShMFAkiRJ\nEgYDSZIkSUwgGCQ5IslFSS5Jctwir/9+kq8M/nwuyb3arkmSJEnSrbUaDJJsB5wAHA4cBByV5MAF\np30beFhV3Rt4NfC2NmuSJEmStKm2ewwOAS6tqsuqaj1wGnDk8AlVdW5V/XDw9Fxgdcs1SZIkSVqg\n7WCwGrhi6PmVLP+L/7OBT7RakSRJkqRNbN91AfOSPAJ4JvCQrmuRJEmStjVtB4OrgDVDz/cfHLuV\nJAcDJwFHVNXcUm+2du3ajY9nZmaYmZkZV52SJElSL83OzjI7O7vZ81JVrRWRZAVwMXAYcA1wHnBU\nVV04dM4a4NPA0VV17jLvVW3W2pYEprDsqWM7T45tPTm29WTYzpNjW0+ObT0Z09rOSaiqLDzeao9B\nVW1I8lzgTJr5DCdX1YVJjm1erpOAlwGrgH9MEmB9VR3SZl2SJEmSbq3VHoNxssdAy7GdJ8e2nhzb\nejJs58mxrSfHtp6MaW3npXoM3PlYkiRJksFAkiRJksFAkiRJEgYDSZIkSRgMJEmSJGEwkCRJkoTB\nQJIkSRIGA0mSJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwkSZIkYTCQJEmShMFAkiRJEgYD\nSZIkSRgMJEmSJGEwkCRJkoTBQJIkSRIGA0mSJEkYDCRJkiRhMJAkSZKEwUCSJEkSBgNJkiRJGAwk\nSZIkYTCQJEmShMFAkiRJEgYDSZIkSRgMJEmSJGEwkCRJkoTBQJIkSRIGA0mSJEkYDCRJkiRhMJAk\nSZKEwUCSJEkSBgNJkiRJTCAYJDkiyUVJLkly3BLnvCnJpUnOT3KftmuSJEmSdGutBoMk2wEnAIcD\nBwFHJTlwwTmPAe5aVXcDjgXe0mZNkiRJkjbVdo/BIcClVXVZVa0HTgOOXHDOkcApAFX1BeD2SfZt\nuS5JkiRJQ9oOBquBK4aeXzk4ttw5Vy1yjiRJkqQWOflYkiRJEtu3/P5XAWuGnu8/OLbwnDtu5hwA\nkoy1uEmZ0rKnju08Obb15NjWk2E7T45tPTm29WT0qZ3bDgZfBH4tyQHANcDTgKMWnHM68CfA+5Ic\nClxfVdcufKOq6lGzS5IkSVuXVoNBVW1I8lzgTJphSydX1YVJjm1erpOq6uNJHpvkm8CNwDPbrEmS\nJEnSplJVXdcgSZIkqWNOPpYkSZJkMJAkSZJkMFBPJLlLkjt1XYckSdK0MhiMUZIrkly+2J+ua+ub\nJP+S5EGDx88ELgAuSPKsbivbNiTZOclOXdfRR0n2SnJ0kpcMnu+XZP+u6+qjJAcmeVmSE4eeH9x1\nXX2TZJ8kuw0er0jyzCTHJPF3EE2tvn5W+59yvP4AOHroz0to9mT4+y6L6qnDgC8NHr8I+G3gEOCl\nnVXUY0lel+SQwePfAdYBc0ke321l/ZLk4cDFwNOBlw0O3w34p86K6qkkTwbOBlbTfF4D7Ab8Q2dF\n9ddHaa5jgL8GXgy8EH82brHlbkh6c7I9ff6sdlWiliX5FeCTVXWfrmvpkyTXV9WeSVYD51XV6sHx\nG6pqj47L650k1wB3raqbknwBeC3wQ+D1VXWvbqvrjyRfBl5cVZ9OMldVK5PcDrisqvbtur4+SXIh\n8LSq+spQW+8AXF1V+3RdX58kmQNWVVUluRJ4EPBj4IKq+tVuq5tug19Q5z0AOAZ4E3AZcADwXOCU\nqjKEjVGfP6vb3uBMcDNw566L6KHzkxxP88H3MYBBSLih06r6a5dBKNgLuEtVfQhgsHmhxudOVfXp\nweP5uzY/w8/qNtwB+OrgcQ397d2y8dsA7Jjk7sAPq+rywTCi3Tqua+pV1VnzjwdD4g6vqquGjn0C\n+CT2zoxbbz+rp/4fsDVJ8qoFh3YBHgt8ooNy+u5ZwF8B64E/Hxx7IPDPnVXUb5ckeTrwa8D/BUiy\nN/CTTqvqn28kObyqPjV07LeBr3VVUI/9J80QolOGjj0NOK+bcnrtE8D7gb2A0wbH7kkz1Fbjsx9N\nT8ywH9MMl9N49faz2qFEY5TknQsO3QicD5xaVTd3UJI0FkkeALyR5o7Is6rqW4OgcERVHb38V2tU\nSQ6lGY/9MeApNL+0Ph44sqq+2GVtfZPkQOBM4DvAocAscHfg0VV1aYel9c5goYJjaG7knFpVP08y\nA/xKVZ227BdrZEneRTNC4dXAlcAdgeOBy6vqmA5L650lPqufADxh2j+rDQaaSkkCPJvmDt8+VXVw\nkofR/KB5f7fVSbddkv1oFjI4ALgCeE9VXdltVf2UZBfgcdzS1h+tqoV3XDUmg+FD+1bVNV3X0keD\nMe5rgSfT9B5cDXwAeGVV2bs7Zgs+qy+n+aye+l4wg8GYDe6C/CFN191VNHdHPtNpUT2U5K+ARwFv\nAN4ymIh8F+ADVXX/bqvrpyQ7AvcA9gYyf7yq/r2zonomyT2r6huLHF/YZa0tlORNVfX8RY6/oape\n0EVNfZVkT+Afgd8D1lfVrkmeABxSVf+72+qkX16S2wPPB+5HM1dm4y/TVfXoruoaB4PBGCV5NvA3\nwNtpVgRYQzMW/mVV9bYua+ubJFcA962q64ZWBAiwrqpWdl1f3yR5CM2dp52APWgmee8OXFFVd+my\ntj5J8m3gsKr6ztCxxwMnuXrLeC21glmSH1TVXl3U1FdJTgPmgFcB3xh8Xu8DnFNVd1v+q/XLSPIo\nmp70O1TV45P8JrCHN3DGK8mZwArgIyyYa1dVJ3dS1Jg4+Xi8XgI8qqq+Mn8gyfuADwEGg/FawS2T\nrObT7W5sOvFK4/F64LVV9fpBEFuV5OXATV0X1jN/DnwqycOr6pokTwJOoBnuojFI8keDh9sPPZ53\nF+C6CZe0LTgM2K+q1icpgKr6fpI7dFxXryR5HvCnNDcnf29w+Cc0y5c+qKu6eupQYO+q+lnXhYyb\nPQZjlOQHNGPc1w8d24lmXWzvQI1RkrfTTIR9IXANzWoXrwd2rKr/1WVtfZTkh8DKqvrFUA/NjsB3\n5veQ0Hik2cn7z4ATaTbOOaKqvrr8V2lUSeaHdj4U+OzQSwVcC7yxqs6deGE9luSbwEMHYXfd4MbC\nGuDMqjqw6/r6Ism3aHocvzv0Ob0C+J6/g4xXko8DL+3jZ7M9BuP1H8A/JDlusOb7rsBrgHM6rquP\nXgS8m2aTrR1oegrOpJnfofH7Ic0QouuBa5LcE/gBrkO+xQYTMoe9G1gFvBx4NHBBku2q6hcTL66H\nquoRAEle7fj2iXk78KEkfwlsl+SBNMNu39JtWb2zO80kerilJ30HmptoGq9nAB8fbPh57fALVbVw\n6fqpYjAYr2Np1mi+YdB7sIomFBzVaVU9VFU3AL+bZF+auRxXVNV/d1xWn32YZk+O9wLvAD5Ds/Tg\nB7ssqid+zqabas1P7j5/8Lhohs9pTIZDwWB+0vCEekPYeP0dzZCWE2l+UX0H8FaaJZA1PmcDLwX+\neujY82k+rzVef02zHOx3aW6azZv6YTgOJRqjJK+nWcv2+wyWCnOZwfFZ5M7qovyh3r4kD6XpLfiU\n7b1lRt09uqoua7uWbclgqcETgYcBew6/VlWGME2dJL8KnEGzctxq4NvAj4DHeeNsvJL8CLh7H5fe\ntcdgvFbQbD3+feBU4D3dltM7i91ZHead1Qmpqs9u/iyNwl/4O/NWmsnzhwFn0QSEtcDHO6ypN5I8\nrKrOHjx+5FLnuVrO+AzmcDwAeAC37M1xnjdvWvFtml7z3rHHYMwGE30eAzydZiWRL9D0InzYjXO2\njHdWu5PkzjRdp/dhwbyCqlrTSVE9keSkqnrO4PGpLBF+q8r5M2M0GO65pqpuTHL9YC+UVTRLaDoh\ndgsl+XpV/cbg8XeWOK1c7ni8kuxAs2LOflX1vsFcR6rqxm4r65ckLwaeBLyZTecYTHXYNRi0KMlB\nNGOy70VzZ+o04BV92BlvazEYG7w3cF15MbcmyeeBbwH/zIIlSqvqrE6K6okkx1fVawaPX7HUeVX1\nyslV1X9JvgfcsapuTvJdmrusN9B8luzeaXE9k2RFVW3ouo6+S3Iv4HTgZmD/qtotyWOBY6rqqd1W\n1y99DrsGgzFLsgfNduR/ABxMs4fBu2m2y/4z4JFVdXB3FfbDYCfNNwNP4ZZVFz4A/GlVreuytj5K\ncgOwp13S47fcMIth034XamuT5AzgHVX1kSRvBe5GM0F2l/mVi7TlBr3oP6b5/Li563r6LMnngLdW\n1alDy5XuClzistIalcFgjJJ8EDicZmWAU4B/Hf4gHEye/aF3o7Zcko8AG2jWeb+MZjzlK2n2MXhi\nl7X1UZKP0vR2/WfXtfTNMneehk39XaitzeDmwnZVtS7JzjQ3bnYD3uBEzfFK8hXgMVV1dde19FmS\nOWBVVdX8fhGD4xsfS5vj5OPxOhd47lI/VAabQ+074Zr66pE0m8nNb0V+YZJnAP7gacd3gU8OAtmt\nru+qenknFfVEVd256xq2UTcBz0gyPG8mwGtxP5Rx+2fgo0neCFzJ0Dwae8LG6rvA/YEvzR9Icgjw\nza4K0vQxGIxRVb1uhHNu2tw5GslFwJ2AC4eOrQEu7qSa/tsV+CjNsK07dlyLNA7vBu5Ns7zjtZs5\nV1vmjwd/r11wvAB7wsbnZcDHkrwF2CnJ8TRt/+xuy9I0cSiRplKSvwGOplkW9gqaX1b/YPD8W/Pn\nVdU7OilQ0lZtMOzizlV1fde19J2Tjydn0AP2HJrhtZcDb6uq/+q2Kk0Tg4GmUpJRdnKsqhppYqdG\nk2R3mlWghneJ/XZ3FUm3zWDc+6Oryt6CFjn5eHKS7Ag8A7gvzfC44SFbDo/TSBxKpKnkqiGTleSe\nNOOE703zw2Z+MzlwQzlNiQUrQJ0C/Ntg3Huv1iHfmlTVhiSXAHvhHLC2DQ+PcwK9bhN7DDS1BquK\n/A6wH80PnI9X1Vy3VfVTklngv4BXAd+hmd/xGprNoNzhW1PBFaC6keQlwNMAJx+3yOFxGgeDgabS\n4M7fh2kmG19GM/H4QOB/VNWnu6ytjwY/cO5QVeuHdondFfi6q+pIWk6fN4Pamjg8TuPgUCJNqxOA\n51TV++cPJHkycCJNQNB4/ZRmRaL1wHVJ1gBzNMMDJGlJ3jyYGIfHaYvZY6CplOR6YK/hlS6SbA9c\nV1V7dldZPyV5P81QrXcl+Vvg8cDNwOVuKCdpc5LsABwK7FdV7xv0OFJVN3ZbWX/YM6NxMBhoKiV5\nM3BpVb1p6NjzgLtV1fO7q6z/Bjt4P51m1YtT/MEuaTlJ7gWcTnMzYf+q2i3JY4Fjquqp3VYnaZjB\nQFMpyeeA36LpLr0KWA3sS7P79PDEtod1UqAkCdj4ef3Wqjo1yVxVrRz0GFxSVau7rk/SLQwGmkpJ\njhnlvKp6d9u1bAuSrAJeDNyHpqdgI8OXpOUMFi9YVVWVZF1VrRoc3/hY0tbByceaSlX17iT7Aoew\n6YZb7nY8fu8FdgLeD9zUcS2Spst3gfsDX5o/kOQQ4JtdFSRpcQYDTaUkTwROpfnBchBwAfAbwOcA\ng8H4PQjKDkxwAAAEdklEQVTYx51LJd0GLwM+luQtwE5Jjgf+GHh2t2VJWmi7rguQbqNXA39UVfcF\nbhz8/RzgP7stq7e+CuzfdRGSpk9VfRQ4HNgH+AzNvjO/W1VndlqYpE3YY6BptaaqPrDg2LtptoF/\ncQf19N2/A59M8k6aNt7IoVuSlpNkR+BJwKNpdqq/imY/lAuq6qedFifpVgwGmlbfS7LvYIfH7yZ5\nIHAdsKLjuvrqocCVwKMWHC8cuiVpef8E3AN4HrfsVP+XNKvJ/VGHdUlawFWJNJWSHAd8s6o+lOQP\ngZOAXwB/X1Uv67Y6SdK8JD8A7lpV1w8dW0XzGe6qRNJWxGCgXkiyBti1qi7supa+SxJuvQrULzos\nR9JWLskFwKOq6uqhY6uBM6vqoO4qk7SQQ4nUC1V1edc19Nngh/gJwMOAPRe87PAtSbeS5JFDT0+l\nmaP0ZpohiXcE/gQ4pYvaJC3NHgNJm5XkDJr9C14DnEUTENYCH6+qt3VYmqStUJLvjHBaVdVdWi9G\n0sgMBpI2azBGeE1V3Zjk+qraczBG+JyqOrDr+iRJ0pZzHwNJo9gA/Hzw+Pok+wA30qwqIkmSesBg\nIGkUXwAeO3j8KeB9wIeBL3VWkSRJGiuHEknarCR70nxezCW5Hc0mcrsBb6yqa7qtTpIkjYM9BpJG\ncRPwwiSXAj8AjqEZWjTXaVWSJGlsXK5U0ijmdy59Ps3OpQcAfwHshzuXSpLUCw4lkrRZ7lwqSVL/\nOZRI0ij+G9hlwbGdAecXSJLUEw4lkrQody6VJGnb4lAiSYty51JJkrYtBgNJkiRJzjGQJEmSZDCQ\nJEmShMFAkiRJEgYDSdKQJP+Q5PlDzz+Z5KSh569L8oLb+N6vSPKicdQpSRo/g4Ekadh/AA8CSBJg\nb+CgodcfBJyzuTdJsqKV6iRJrTEYSJKGncMgGNAEgq8DP0py+yQ7AgcCX07yf5J8LclXkjwFIMnD\nk5yd5N+ACwbH/jLJxUnOBu4x/02SPD/JBUnOT/LeSf4DJUmLc4MzSdJGVXVNkvVJ9ueW3oHVwAOB\nG4CvAY8HDq6qeyW5A/DFJGcN3uK+wEFVdXmS+wFPAQ4GdgT+C/jS4LzjgDtV1foke0zq3ydJWpo9\nBpKkhc4BHkwTDD4PnDv0/D+AhwD/AlBV3wNmgQcMvva8qrp88PihwEeq6uaq+hFw+tD3+Arw3iRP\nBza0+q+RJI3EYCBJWmh+ONFv0AwlOpemx+CBLD6/IEOPbxzxe/wOcAJwP5oeB38eSVLH/CCWJC10\nDvA4YF015oA9uSUYfBZ4apLtkuxD0zNw3iLvczbwxCQ7JdmdZgjSvDVVdRbwUmAPYLf2/jmSpFE4\nx0CStNDXgL2A9yw4tktVrQM+kuRQmuFAvwD+vKq+l+TXh9+kqr6c5H3AV4FrGYSHJNsD7xnMLQjw\nxqq6oe1/lCRpeamqrmuQJEmS1DGHEkmSJEkyGEiSJEkyGEiSJEnCYCBJkiQJg4EkSZIkDAaSJEmS\nMBhIkiRJwmAgSZIkCfj/O+KPYnw01S4AAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"hierarchy.dendrogram(linkage, color_threshold=0.3, leaf_label_func=label, leaf_rotation=90)\n",
"plt.xlabel(\"Words\")\n",
"plt.ylabel(\"Distance\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You may want to explore the options for the dendrogram here:\n",
"\n",
"http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.dendrogram.html"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3. Team Reflections"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Write about what you have found. You can write much here. You can explain the experiment, results found, where they agree with Elman, or disagree with his results."
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbgrader": {
"grade": true,
"grade_id": "team-analysis",
"locked": false,
"points": 100,
"solution": true
}
},
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 4. Reflections"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As per usual, please reflect deeply on this week's lab. What was challenging, easy, or surprising? Connect the topics onto what you already know."
]
},
{
"cell_type": "markdown",
"metadata": {
"nbgrader": {
"grade": true,
"grade_id": "reflections",
"locked": false,
"points": 50,
"solution": true
}
},
"source": []
}
],
"metadata": {
"celltoolbar": "Create Assignment",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}